Fine-Grained Egocentric Hand-Object Segmentation: Dataset, Model, and Applications

Egocentric videos offer fine-grained information for high-fidelity modeling of human behaviors. Hands and interacting objects are one crucial aspect understanding a viewer’s behaviors intentions. We provide labeled dataset consisting 11,243 egocentric images with per-pixel segmentation labels hands being interacted during diverse array daily activities. Our is the first to label detailed hand-object contact boundaries. introduce context-aware compositional data augmentation technique adapt out-of-distribution YouTube video. show that our robust model can serve as foundational tool boost or enable several downstream vision applications, including hand state classification, video activity recognition, 3D mesh reconstruction interactions, inpainting foregrounds in videos. Dataset code available at: https://github.com/owenzlz/EgoHOS .